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DoD’s  Software  Challenge 


“DoD  estimates  that  it  spends  about  40% 
of  its  RDT&E  budget  on  software  -  $21 B  for  FY2003”  -  GAO 


F/A-22 


SBIRS-High 


“[Software]  continues  to  grow  in  importance  in  our  weapons 
systems  -  and  remains  a  significant  contributor  to  program 
cost,  schedule  and  performance  shortfalls.”  -  Pete  Aldridge 
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Today’s  Development  Challenges 


Huge  system/software  engineering  endeavors  in 
aircraft,  space  vehicles,  command  and  control, 
ground  infrastructure,  battle  management,  etc 

•  Several  million  SLOC  programs 

•  “Hybrid”  systems  combining  legacy  re-use,  COTS, 
new  development 

•  Multi-contractor  teams  using  different  processes; 
Dispersed  engineering  &  development  locations 

•  New  technologies/products  -  rapid  change  and 
evolution;  are  they  mature;  obsolescence 

•  Business/operational  needs  change  -  often  faster 
than  full  system  capability  can  be  implemented 

•  Skillset  Shortfalls;  Cost  and  schedule  constraints 

•  Demands  for  increased  integration,  interoperability, 
system  of  system  capabilities 
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Increasing  System  Complexity 
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Capability  Delivered  in  Software 
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Software  is  Even  in  Bullets! 


150K  SLOC  -  Weapon 
2K  SLOC  -  Ammunition 
Ada 


Wide  Area  Munition 


Infantry  Combat  Weapon 


130K  SLOC 

Ada,  C++,  C,  Assembly 
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And  Software  Connects  Systems... 
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Operational  Insight 

New  Capabilities 


Environment 
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Introduction:  Current  Environment 


Providing  enhanced  capability  to  the  warfighter  is  a  complex  and 
conflict-ridden  endeavor. 

Operational  forces  demand  war-winning  systems.  They  need 
evolutionary  enhancements  to  existing  systems  to  maintain  a  cutting 
edge  on  the  battlefield. 

Acquirers  need  to  maintain  cost,  schedule,  and  technical  baselines  to 
uphold  their  duty  as  stewards  of  the  taxpayers’  money  and  to  satisfy 
oversight  requirements. 

Contractors  need  to  win  contracts  to  stay  in  business  and  sustain  the 
industry  base. 

Underpinning  these  conflicts  is  an  ever-increasing  demand  on 
systems  and  software  engineering  to  solve  the  complexities  of  an 
interconnected  battlespace. 
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The  Acquirer’s  Job 


Operational 
Need 


What  are  the  key  activities  you 
perform  when  you  acquire  systems? 

Requirements  Risk  Program 

Management  Management  Integration 

Configuration  Verification  Project 

Management  gnd  Va|idation  Planning 


Need  to  counter  these  attitudes: 


“I" 


I'd  rather  have  it  wrong  than  have  it  late.”  —  Industry  senior  manager 


•  “Ad  hoc,  catch  as  you  can... that’s  our  motto.”  —  PMO 

•  “We  do  not  work  problems  until  they’re  unrecoverable.”  -  PMO 

•  “I  don’t  want  an  ATAM  [to  reveal  problems]  on  my  watch.”  —  PMO 
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Visibility  into  the  Team’s  Capability 


Operational 

Need 


Acquirer  CMMI-AM  or  CMMI-ACQ 
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The  “Team” 


CMMI  Math:  3  +  4  +  5  +  ?  =  ? 
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DoD’s  Problem  Statement 

Many  DoD  contractors  advertise  high  levels  of 
process  capability  or  organizational  maturity  as 
measured  by  either  the  Continuous  or 
Staged  representations  of  Capability  Maturity 
Model  Integration,  yet  from  the  perspective 
of  acquisition  program  managers  on  some 
high  visibility  individual  programs,  strong 
systems  engineering  and  project  management 
practices  still  appear  to  be  lacking. 
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Example 


Large  DoD  program  with  multiple,  geographically  dispersed 
engineering  locations. 

Multi-contractor  teams  (10+)  using  different  processes. 
Several  million  lines  of  code. 

Systems  engineering  challenges. 

Combination  of  legacy,  re-use,  COTS  integration  and  new 
development. 

All  contractor  sites  are  Maturity  Level  3  or  higher. 

18  months  after  contract  award,  the  program  office 
conducted  a  CMMI  “Class  B”  appraisal  on  the  team. 
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Characterizing  Results 
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Project  Mgmt  Engineering  Process  Support 

Processes  Mgmt  Processes 


□  Number  of  Strengths  ■  Number  of  Weaknesses 


Project  Mgmt  Processes: 

-  Project  Planning 

-  Project  Monitoring  &  Control 

-  Integrated  Project  Mgmt 

-  Risk  Management 

Engineering  Processes 

-  Requirements  Mgmt 

-  Requirements  Definition 

-  Technical  Solution 

-  Product  Integration 

-  Verification  (Peer  Reviews) 

Support  Processes 

-  Measurement  &  Analysis 

-  Product  and  Process  Quality 
Assurance 

-  Configuration  Mgmt 

-  Decision  Analysis 

Process  Mgmt 

-  Organizational  Process  Focus 

-  Organizational  Process  Definition 
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Issues  Identified  -  Program  Management 

Lack  of  project  plans  or  having  only  incomplete,  conflicting  or  out 
of  date  project  plans 

Ineffective  use  of  Integrated  Master  Schedule  as  basis  for 
planning/tracking  status  across  program 

Undefined  engineering  and  management  processes  on  program 

Inability  to  track  and  manage  actions  to  closure 

Inadequate  cost  estimation  processes,  methods,  data  and  tools 

Inadequate  staffing  and  training  project  personnel 

Tracking  dependencies  between  or  across  teams  not  defined 

Managing  project  data  ad  hoc 

Inability  to  proactively  identify  and  manage  risks 
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Issues  Identified  -  Engineering 


Lack  of  understanding  of  the  program’s  requirements 

Inability  to  trace  requirements  to  architecture/design  or  to  test 
plans/procedures 

Poor  linkage  of  functional  and  performance  requirements 

Inconsistent  requirements  management  at  different  levels 

No  criteria  for  making  architectural/design  decisions  among 
alternatives 

Not  capturing  entire  technical  data  package  (requirements,  design 
and  design  rationale,  test  results,  etc) 

Efficiency  of  design  process/methods  in  question 

Late  definition  of  integration  and  test  procedures 
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=.  GarnegieMellan 
—  Software  Engineering  Institute 

Issues  Identified  -  Support  Processes 

Difficult  to  identify  items  in  configuration  management  baselines 

Lack  of  ability  to  manage  individual  “versions”  in  incremental  development 

Inability  to  effectively  managing  changes  to  work  products  throughout 
lifecycle 

Not  conducting  audits  to  establish/ensure  integrity  of  baselines  throughout 
incremental  engineering  and  development 

Inefficient  change  management  process  (cycle  time,  volume  of  changes) 

Quality  Assurance  audits  of  products  and  processes  not  consistent 

QA  involvement  in  system  and  software  engineering  processes  not 
consistent 

No  metrics  to  manage  engineering  activities  (outside  of  cost/schedule  data) 
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CMMI  vl.2  -  Part  of  the  Solution! 

Increasing  the  integrity  and  credibility  of  the  model 


Emphasizing  project  “start-up”  and  process  deployment 
Increasing  the  integrity  and  credibility  of  the  appraisal  process 
“Raising  the  bar”  for  SCAMPI  Lead  Appraisers 


CMMI  is  a  key  enabler  as  the  DoD 
acquires  increasingly  complex 
capabilities  and  systems 
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Contact  Information 

Rick  Barbour 

Chief  Engineer  Navy,  Acquisition  Support  Program 

Software  Engineering  Institute 
4500  Fifth  Ave. 

Pittsburgh,  PA  15213-3890 
(412)  268-7157 

reb@sei.cmu.edu 

Acquisition  Support  Program: 

Director:  Brian  Gallagher 

bg@sei.cmu.edu 

Air  Force::  John  Foreman 

jtf@sei.cmu.edu 

Army:  Cecilia  Albert 

cca@sei.cmu.edu 

Intelligence  Community:  Rita  Creel 

rc@sei.cmu.edu 

Civil  Agencies:  Steve  Palmquist 

msp@sei.cmu.edu 

http://www.sei.cmu.edu/programs/acquisition-support/ 
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